IEE E P ro of 1 MPI - FAUN : An MPI - Based Framework 2 for Alternating - Updating Nonnegative

5 Abstract—Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factorsW andH, for the 6 given input matrix A, such thatA WH. NMF is a useful tool for many applications in different domains such as topic modeling in text 7 mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining 8 community, there is a lack of efficient parallel algorithms to solve the problem for big data sets. The main contribution of this work is a 9 new, high-performance parallel computational framework for a broad class of NMF algorithms that iteratively solves alternating 10 non-negative least squares (NLS) subproblems forW andH. It maintains the data and factor matrices in memory (distributed across 11 processors), uses MPI for interprocessor communication, and, in the dense case, provably minimizes communication costs (under mild 12 assumptions). The framework is flexible and able to leverage a variety of NMF and NLS algorithms, including Multiplicative Update, 13 Hierarchical Alternating Least Squares, and Block Principal Pivoting. Our implementation allows us to benchmark and compare 14 different algorithms on massive dense and sparse data matrices of size that spans from few hundreds of millions to billions. We 15 demonstrate the scalability of our algorithm and compare it with baseline implementations, showing significant performance 16 improvements. The code and the datasets used for conducting the experiments are available online.